The present invention relates to a method of automatically creating an image database that can be interrogated by its semantic content, the database being created from initial images that are not indexed.
The invention also relates to a method of indexing and searching images by means of their semantic content.
The object of the invention is to make it possible to interrogate image bases by content, in a manner that is ergonomic and intuitive for the user.
A first possible solution for interrogating an image base might consist in reusing the techniques devised for searching text information, using images that have previously been annotated manually. That would consist in formulating a textual query in order to obtain in return a set of images whose textual descriptions (the fruit of prior indexing) match the query more or less well. Such an approach would indeed give access to the semantic content of images, but unfortunately it implies indexing being performed manually. That type of procedure is thus relatively lengthy, tedious, and inconceivable in the presence of a large number of images. Furthermore, that style of indexing would be effective in use only for bases where the indexers and the users possess the same level of expertise. Furthermore, given the time necessary for generating the catalog of the images, it is illusory to imagine that the images could be described with sufficient accuracy or objectivity, particularly since the target of a future search is not necessarily known during the indexing process. Important elements in the image might then be ignored. Furthermore, an element such as a building, a vehicle, or a character when described in terms of the category of which that element forms a part, would be identified only after visually examining the image.
Another possible solution might consist in reusing the work of computer vision researchers using image analysis, processing, coding, and compression. To do this, it is necessary to use systems capable of responding to the visual properties of images (texture, shape, color). That type of approach would indeed enable automatic indexing to be performed in objective manner and without a priori knowledge. However that approach would come up against the semantic barrier since two images can be perceptibly similar without that making them cognitively similar, and similarly two images that are cognitively similar can be utterly different perceptually.
With that style of approach known as “search by example”, there already exist various products that accept as a question a sample that resembles the image sought by the user. Various solutions have thus been proposed:                a first solution consists in looking for an object by describing its outline or its texture. However to enable that type of approach to work, it is necessary to analyze the images digitally and to extract the corresponding information from them. Unfortunately, there does not exist any method of extracting the contour or the texture of an object which is effective under all circumstances, and furthermore, those methods fail when objects are masked in part, are in the shade, or when the sample and the images in the database are subject to different kinds of lighting; and        a second possible approach consists in comparing the sample with various portions of the images; a comparison is positive when the points of an image fragment have the same hue as the points of the sample. In addition to the lengthy computation time due to the number of operations needed, the sample image must be taken under conditions that are similar to those which applied when the images in the database were taken (lighting, distance, orientation), which limits the impact of that type of approach.        
Another drawback results from the fact that the user must possess a sample that is representative of the image being sought. Unfortunately, there exist numerous circumstances when the user does not possesses such a sample, the user merely having an idea of what to look for.